{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "zero_padding_in_cnn.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMxtTFhW9SHmiNdgfO/llBl",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SummerLife/EmbeddedSystem/blob/master/MachineLearning/gist/zero_padding_in_cnn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ecGIhp0S52Xz",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "bb46f18d-ba3e-4d4b-8d62-e290c06e051c"
      },
      "source": [
        "import keras\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Activation\n",
        "from keras.layers.core import Dense, Flatten\n",
        "from keras.layers.convolutional import *"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using TensorFlow backend.\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FHeTwYbfBr6k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model_valid = Sequential([\n",
        "    Dense(16, input_shape=(20,20,3), activation='relu'),\n",
        "    Conv2D(32, kernel_size=(3,3), activation='relu', padding='valid'),\n",
        "    Conv2D(64, kernel_size=(5,5), activation='relu', padding='valid'),\n",
        "    Conv2D(128, kernel_size=(7,7), activation='relu', padding='valid'),\n",
        "    Flatten(),\n",
        "    Dense(2, activation='softmax')\n",
        "])"
      ],
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P1zzKtbrCQK5",
        "colab_type": "text"
      },
      "source": [
        "We’ve specified that the input size of the images that are coming into this CNN is 20 x 20, and our first convolutional layer has a filter size of 3 x 3, which is specified in Keras with the kernel_size parameter. Then, the second conv layer specifies size 5 x 5, and the third, 7 x 7.\n",
        "\n",
        "With this model, we’re specifying the parameter called padding for each convolutional layer. We’re setting this parameter equal to the string 'valid'. Remember from earlier that, valid padding means no padding.\n",
        "\n",
        "This is actually the default for convolutional layers in Keras, so if we don’t specify this parameter, it’s going to default to valid padding. Since we’re using valid padding here, we expect the dimension of our output from each of these convolutional layers to decrease."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WWqNeaEEBuNO",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        },
        "outputId": "e0b011d2-9ad2-4c9e-c8b1-434e7b24cae7"
      },
      "source": [
        "model_valid.summary()"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_1 (Dense)              (None, 20, 20, 16)        64        \n",
            "_________________________________________________________________\n",
            "conv2d_1 (Conv2D)            (None, 18, 18, 32)        4640      \n",
            "_________________________________________________________________\n",
            "conv2d_2 (Conv2D)            (None, 14, 14, 64)        51264     \n",
            "_________________________________________________________________\n",
            "conv2d_3 (Conv2D)            (None, 8, 8, 128)         401536    \n",
            "_________________________________________________________________\n",
            "flatten_1 (Flatten)          (None, 8192)              0         \n",
            "_________________________________________________________________\n",
            "dense_2 (Dense)              (None, 2)                 16386     \n",
            "=================================================================\n",
            "Total params: 473,890\n",
            "Trainable params: 473,890\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v-65s-W1CDNk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "model_same = Sequential([\n",
        "    Dense(16, input_shape=(20,20,3), activation='relu'),\n",
        "    Conv2D(32, kernel_size=(3,3), activation='relu', padding='same'),\n",
        "    Conv2D(64, kernel_size=(5,5), activation='relu', padding='same'),\n",
        "    Conv2D(128, kernel_size=(7,7), activation='relu', padding='same'),\n",
        "    Flatten(),\n",
        "    Dense(2, activation='softmax')\n",
        "])"
      ],
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0vZoji4PCroQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        },
        "outputId": "511ec0d7-c8b4-4290-edb4-930f66429c04"
      },
      "source": [
        "model_same.summary()"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_2\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_3 (Dense)              (None, 20, 20, 16)        64        \n",
            "_________________________________________________________________\n",
            "conv2d_4 (Conv2D)            (None, 20, 20, 32)        4640      \n",
            "_________________________________________________________________\n",
            "conv2d_5 (Conv2D)            (None, 20, 20, 64)        51264     \n",
            "_________________________________________________________________\n",
            "conv2d_6 (Conv2D)            (None, 20, 20, 128)       401536    \n",
            "_________________________________________________________________\n",
            "flatten_2 (Flatten)          (None, 51200)             0         \n",
            "_________________________________________________________________\n",
            "dense_4 (Dense)              (None, 2)                 102402    \n",
            "=================================================================\n",
            "Total params: 559,906\n",
            "Trainable params: 559,906\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cYzoCYF1C3ac",
        "colab_type": "text"
      },
      "source": [
        "We can see again that we’re starting out with our input size of 20 x 20, and if we look at the output shape for each of the convolutional layers, we see that the layers do indeed maintain the original input size now.\n",
        "\n",
        "This is why we call this type of padding same padding. Same padding keeps the input dimensions the same."
      ]
    }
  ]
}